Emergence of Visual Saliency from Natural Scenes via Context-Mediated Probability Distributions Coding

Visual saliency is the perceptual quality that makes some items in visual scenes stand out from their immediate contexts. Visual saliency plays important roles in natural vision in that saliency can direct eye movements, deploy attention, and facilitate tasks like object detection and scene understanding. A central unsolved issue is: What features should be encoded in the early visual cortex for detecting salient features in natural scenes? To explore this important issue, we propose a hypothesis that visual saliency is based on efficient encoding of the probability distributions (PDs) of visual variables in specific contexts in natural scenes, referred to as context-mediated PDs in natural scenes. In this concept, computational units in the model of the early visual system do not act as feature detectors but rather as estimators of the context-mediated PDs of a full range of visual variables in natural scenes, which directly give rise to a measure of visual saliency of any input stimulus. To test this hypothesis, we developed a model of the context-mediated PDs in natural scenes using a modified algorithm for independent component analysis (ICA) and derived a measure of visual saliency based on these PDs estimated from a set of natural scenes. We demonstrated that visual saliency based on the context-mediated PDs in natural scenes effectively predicts human gaze in free-viewing of both static and dynamic natural scenes. This study suggests that the computation based on the context-mediated PDs of visual variables in natural scenes may underlie the neural mechanism in the early visual cortex for detecting salient features in natural scenes

Low level constraints on dynamic contour path integration

Contour integration is a fundamental visual process. The constraints on integrating discrete contour elements and the associated neural mechanisms have typically been investigated using static contour paths. However, in our dynamic natural environment objects and scenes vary over space and time. With the aim of investigating the parameters affecting spatiotemporal contour path integration, we measured human contrast detection performance of a briefly presented foveal target embedded in dynamic collinear stimulus sequences (comprising five short 'predictor' bars appearing consecutively towards the fovea, followed by the 'target' bar) in four experiments. The data showed that participants' target detection performance was relatively unchanged when individual contour elements were separated by up to 2° spatial gap or 200ms temporal gap. Randomising the luminance contrast or colour of the predictors, on the other hand, had similar detrimental effect on grouping dynamic contour path and subsequent target detection performance. Randomising the orientation of the predictors reduced target detection performance greater than introducing misalignment relative to the contour path. The results suggest that the visual system integrates dynamic path elements to bias target detection even when the continuity of path is disrupted in terms of spatial (2°), temporal (200ms), colour (over 10 colours) and luminance (-25% to 25%) information. We discuss how the findings can be largely reconciled within the functioning of V1 horizontal connections

The color tuning of independent components of natural scenes matches V1 simple cells

The early visual system is hypothesized to reduce the redundancy inherent to natural images. To test this theory, Independent Component Analysis (ICA) has been applied to monochromatic natural images, yielding independent component (IC) filters, which maximally reduce redundancy. The receptive fields of these ICs strongly resemble V1 receptive fields, suggesting that redundancy reduction is a good model for V1. ICA has more recently been applied to colored natural images. The resulting ICs appear chromatically similar to visual neurons (Hoyer and Hyvarinen 2000, Wachtler, Lee and Sejnowski 2001). However, no quantitative analysis of the relationship between color sensitivity and luminance sensitivity has been done. We do so by 'stimulating' ICs with colored sinusoidal gratings of differing contrast, comparing the results against physiological data (Lennie et al. 1990). We find that the color tuning of ICs strongly resembles the color tuning of V1 simple cells, but not the unoriented cells found in V1 color blobs. Both simple cells and ICs are dominated by red-green opponent filters and blue-yellow opponent filters. Red-green opponency is not perfectly balanced, and so those filters respond more strongly to changes in luminance than purely chromatic changes. Many blue-yellow opponent filters, however, show significant response to purely chromatic changes. These properties are observed in both simple cells and ICs, providing further evidence for the view that simple cells are redundancy reducing filters. The mismatch between the color tuning of ICs and unoriented blob cells suggests that V1 color analysis cannot be entirely modelled by straightforward ICA. This suggests that blob cells may be part of a separate pathway that does not simultaneously reduce spatial and chromatic redundancy, but perhaps only local color redundancy

Independent components of color natural scenes resemble V1 neurons in their spatial and color tuning.

It has been hypothesized that mammalian sensory systems are efficient because they reduce the redundancy of natural sensory input. If correct, this theory could unify our understanding of sensory coding; here, we test its predictions for color coding in the primate primary visual cortex (V1). We apply independent component analysis (ICA) to simulated cone responses to natural scenes, obtaining a set of colored independent component (IC) filters that form a redundancy-reducing visual code. We compare IC filters with physiologically measured V1 neurons, and find great spatial similarity between IC filters and V1 simple cells. On cursory inspection, there is little chromatic similarity; however, we find that many apparent differences result from biases in the physiological measurements and ICA analysis. After correcting these biases, we find that the chromatic tuning of IC filters does indeed resemble the population of V1 neurons, supporting the redundancy-reduction hypothesis

A natural approach to studying vision

An ultimate goal of systems neuroscience is to understand how sensory stimuli encountered in the natural environment are processed by neural circuits. Achieving this goal requires knowledge of both the characteristics of natural stimuli and the response properties of sensory neurons under natural stimulation. Most of our current notions of sensory processing have come from experiments using simple, parametric stimulus sets. However, a growing number of researchers have begun to question whether this approach alone is sufficient for understanding the real-life sensory tasks performed by the organism. Here, focusing on the early visual pathway, we argue that the use of natural stimuli is vital for advancing our understanding of sensory processing